Benchmark multiple different pipelines running in parallel #18
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jim-wang-intel
changed the title
jim-wang-intel
changed the title
jim-wang-intel
force-pushed
the
parallel-pipelines
branch
from
164181e
to
57f4305
Compare
This was referenced Nov 7, 2024
Closed
…g different pipelines at the same time Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
jim-wang-intel
force-pushed
the
parallel-pipelines
branch
from
4ddd45a
to
90a78fa
Compare
jim-wang-intel
requested review from
antoniomtz,
seanohair22,
brian-intel and
NeethuES-intel
jim-wang-intel
changed the title
|
|
… added loss-prevention container Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
seanohair22
approved these changes
Nov 15, 2024
antoniomtz
requested changes
Nov 15, 2024
src/docker-compose-2-clients.yml
Outdated
| volumes: | ||
| - ${RETAIL_USE_CASE_ROOT:-..}/performance-tools/sample-media:/home/pipeline-server/sample-media | ||
|
|
||
| OvmsClientGst1: |
There was a problem hiding this comment.
This container isn't OpenVINO Model Server, we should change the name of it. Maybe GstClient
benchmark.md
Outdated
| @@ -38,6 +38,45 @@ make benchmark-stream-density | |||
| !!! Note | |||
| For more details on how this works, you can check the documentation of performance-tools in [Benchmark Stream Density for CV Pipelines](https://github.com/intel-retail/documentation/blob/main/docs_src/performance-tools/benchmark.md#benchmark-stream-density-for-cv-pipelines) section. | |||
|
|
|||
| ### Benchmark for multiple pipelines in parallel | |||
|
|
|||
| There is an example docker-compose file under src/ directory, named `docker-compose-2-clients.yml` that can be used to show case both of benchmarks of parallel running pipelines and stream density benchmarks of running pipelines. This docker-compose file contains two different running pipelines: one is running yolov5s pipeline and the other one is yolov8 region of interests pipeline. Follow the follow command examples to do the benchmarks: | |||
There was a problem hiding this comment.
Suggested change
| There is an example docker-compose file under src/ directory, named `docker-compose-2-clients.yml` that can be used to show case both of benchmarks of parallel running pipelines and stream density benchmarks of running pipelines. This docker-compose file contains two different running pipelines: one is running yolov5s pipeline and the other one is yolov8 region of interests pipeline. Follow the follow command examples to do the benchmarks: | |
| There is an example docker-compose file under src/ directory, named `docker-compose-2-clients.yml` that can be used to show case both of benchmarks of parallel running pipelines and stream density benchmarks of running pipelines. This docker-compose file contains two different running pipelines: one is running yolov5s pipeline and the other one is yolov8 region of interests pipeline. Use the following command examples to do the benchmarks: |
There was a problem hiding this comment.
thanks, fixed.
Signed-off-by: Jim Wang <yutsung.jim.wang@intel.com>
antoniomtz
approved these changes
Nov 15, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Checklist
What are you changing?
This PR demonstrate the possibility to do the benchmarks for two (or multiple) different purposes of CV pipelines running in parallel. For more details, check this quick documentation.
Issue this PR will close
close: #issue_number
Anything the reviewer should know when reviewing this PR?
Test Instructions if applicable
make update-submodulesmake DOCKER_COMPOSE=docker-compose-2-clients.yml BENCHMARK_DURATION=90 benchmarknote that BENCHMARK_DURATION increase usually to a bigger value as more pipelines running in parallel in this docker-compose example (docker-compose-2-clients.yml ) tend to slow down the system and so need more time for pipelines to be stabilized. For my NUC box, i need to increase from the default value 45 to at least 90 to gather more meaningful benchmark summary result.
here are some sample outputs you will expect to see:
make clean-benchmark-resultsso that we have a clean start.
make DOCKER_COMPOSE=docker-compose-2-clients.yml BENCHMARK_DURATION=90 TARGET_FPS="10.95 2.95" CONTAINER_NAMES="gst1 gst2" benchmark-stream-densityit should runs ok but this will take a while (about 10-20 minutes), and you should see the final results for the stream density on this two running parallel pipelines (one service is running yolov5s pipeline, and the other is running yolvo8_roi.sh pipeline).
Example console output:
If the there are associated PRs in other repositories, please link them here (i.e. intel-retail/loss-prevention )